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SUMMARY  PAGE 


THE  PROBLEM 

To  delineate  selected  test  score  cut-off  points  which  effectively 
identify  those  non-rated  submariner  volunteers  with  a high  likelihood 
of  failing  the  Basic  Submarine  School. 

FINDINGS 

Based  upon  the  Basic  Test  Battery  distributions  obtained  from  a 
sample  of  955  non-rated  submariner  candidates,  attainment  of  a 
high-school  diploma  was  a most  crucial  selection  standard.  Within 
the  high-school  drop-outs  (42%  of  the  input),  an  ECOP  (Experimental 
Cut-off  Point)  either  at  GCT=50  or  GCT+Mech=100,  would  reduce  the 
false  positive  rate  (proportion  screened  out  who  would  have  passed) 
from  51%  to  40%.  By  extrapolation,  it  was  estimated  that  a GCT=33 
identifies  an  ECOP  below  which  100%  attrition  within  the  high-school 
drop  group  would  occur. 

APPLICATIONS 

This  paper  presents  a simplified  method  for  pinpointing  maxi- 
mally effective  test  score  levels  that  identify  non-rated  men  with 
high  and  low  odds  of  graduating  from  Basic  Submarine  School,  thus 
providing  a means  by  which  significant  amounts  of  training  funds  may 
be  saved. 


ADMINISTRATIVE  INFORMATION 

This  investigation  was  conducted  as  part  of  Bureau  of  Medicine 
and  Surgery  Research  Unit  MF51.  524. 022-0004  - Environmental 
Factors  during  Extended  Missions.  The  present  report  is  Number  1 
on  this  work  unit.  It  was  submitted  for  review  on  16  December  1974, 
approved  for  publication  on  18  December  1974  and  designated  as 
NavSubMedRschLab  Report  No.  797. 

This  study  was  requested  by  the  Commanding  Officer,  Naval 
Submarine  School,  and  was  partially  funded  by  the  Chief  of  Naval 
Education  and  Training. 

PUBLISHED  BY  THE  NAVAL  SUBMARINE  MEDICAL  RESEARCH  LABORATORY 

ii 


1 

| 


ABSTRACT 


The  goals  of  this  study  were  to  identify  the  most  effective  cut- 
off points  on  the  distributions  of  selected  Basic  Test  Battery  (BTB) 
score  combinations  obtained  from  a sample  of  non-rated  volunteers 
for  Basic  Submarine  School.  Based  upon  the  disparities  in  the 
cumulative  percentage  BTB  score  distributions  obtained  from  955 
non-rated  submariner  candidates,  optimal  ECOP's  (Experimental 
Cut-off  Points)  were  delineated  for  GCT,  GCT+MECH,  GCT+MECH+ 

SP,  ARI+MECH+SP  and  for  SP  (Shop  Practices)  test  alone.  Capable 
of  being  applied  by  a middle-level  clerk,  the  analytical  techniques 
provided  estimates  of  the  maximum  COPV  (Coefficient  of  Practical 
Validity),  an  index  of  selection  efficiency,  as  well  as  an  estimate  of 
the  False  Positive  (FP)  and  false  negative  losses  for  several  ECOP's 
for  each  test  score  or  score  combination.  Taking  into  account  the 
90%  requisite  selection  ratio  imposed  at  this  time,  some  of  the  major 
findings  were:  (1)  Level  of  formal  education  is  a crucial  moderator 
variable,  the  most  effective  ECOP  being  high  school  graduate  versus 
dropout;  (2)  the  attrition  rate  (AR)  was  49%  for  H.S.  drop-outs  as 
compared  to  21%  for  H.S.  graduates;  (3)  the  data  argue  that  there  are 
no  usefully  effective  ECOP's  for  the  58%  with  a high  school  diploma 
as  accepting  all  of  this  education  group  yields  an  AR=20%;  (4)  for  the 
H.S.  drops,  GCT=50,  or  GCT+MECH=100  are  equally  effective  but 
both  yield  an  FP  = 40%  which  compares  to  51%  within  all  H.S.  drop- 
outs . A brief  dictionary  of  relevant  selection  terminology  is  included 
as  an  appendix. 
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A SIMPLIFIED  PROCEDURE  FOR  IDENTIFICATION  OF  OPTIMAL  TEST  SCORE 
CUT-OFF  POINTS  FOR  NON-RATED  SUBMARINER  CANDIDATES 


INTRODUCTION 

The  problem  to  which  this  study  is 
addressed  originated  in  a conference 
held  on  30  November  1973  at  the  Bureau 
of  Naval  Personnel,  the  broad  objective 
of  this  conference  having  been  a deline- 
ation of  the  problems  related  to  the 
current  shortages  and  desirable  "qual- 
ity mixes"  of  non-rated,  non-designated 
candidates  for  the  Submarine  Force. 
Stated  in  an  official  letter  from 
CHNAVPERS*,  one  outcome  of  this 
conference  was  the  recommendation 
that  the  selection  efficiency  of  the  GCT, 
MECH,  ARI  and  SP  as  well  as  selected 
combinations  of  these  tests  be  deter- 
mined for  the  non-rated  input  to  Basic 
Enlisted  Submarine  School.  Further,  a 
letter  dated  15  January  from  the  Com- 
manding Officer  of  the  Naval  Submarine 
School  to  the  Commanding  Officer  of  the 
Naval  Submarine  Medical  Center2  re- 
quested that  "the  NAVSUBMEDCEN 
conduct  a study  to  determine  if  a mini- 
mum floor  on  GCT  should  be  estab- 
lished, and  what  it  should  be,  as  well 
as  investigating  the  validity  of  the  Shop 
Practice  criteria  (9-point  waiver  al- 
lowed)", op.  cit.,  p.  3.  Upon  receipt 
of  Project  Order  P04-005  providing 
funding  from  Chief  of  Naval  Education 
and  Training  Support  on  17  April  1974, 
the  study  was  organized  and  data  col- 
lection begun. 

METHODS  AND  PROCEDURE 

The  necessary  first  step  in  the  de- 
termination of  the  efficacy  of  any  per- 
sonnel selection  program  is  to  ascertain 


empirically  the  combinations  of  bio- 
graphical information,  test  scores  and 
other  observational  data  which  will 
maximize  the  predictive  efficiency 
(validity)  in  terms  of  the  available  eval- 
uative standards  (criteria) . There  are 
two  general  statistical  approaches  to 
identification  of  the  most  useful  predic- 
tor variable  combination:  First,  the 
Multiple  Correlation  Technique  most 
often  involving  the  computational  proce- 
dure commonly  called  the  Wherry- 
Doolittle  Technique5  and  second,  the 
Multiple  Cutting-Score  (M  C-S)  method 
of  which  there  are  several  computa- 
tional modes4,5,6.  Since  for  more  than 
three  decades  both  of  these  approaches 
have  been  periodically  applied  to  the 
submariner  selection  situation  and  since 
the  statistical  techniques  used  in  the 
present  study  are  a variation  of  the 
M C-S  method,  some  brief  statements 
regarding  the  pros  and  cons  of  each  will 
be  presented. 

Prior  to  the  advent  of  the  computer, 
the  Multiple  Correlation  Technique 
(MCT) , since  it  requires  first  the  com- 
putation of  a square  matrix  of  correla- 
tion coefficients  (matrix  size  equal  to 
the  number  of  predictor  variables,  "n") 
and  then  the  solution  of  "n"  simultane- 
ous linear  equations , resulted  in  pro- 
hibitive amounts  of  computational  labor, 
particularly  so  if  more  than  4-6  vari- 
ables were  included.  The  outcome 
however  provided  so-called  Beta 
weights,  usually  nonintegers , as 
coefficients  for  each  predictor  variable 
indicating  its  relative  importance  for 
predicting  individual  differences  in  the 
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so-called  criterion  variable,  in  the 
present  study,  graduation  or  failing 
Basic  Submarine  School . Hence  each 
person  may  be  classified  in  terms  of  a 
composite  score  from  which  can  be  ob- 
tained estimates  of  his  success  prob- 
ability. One  Navy  researcher  labels 
this  index.  Odds  for  Effectiveness. 7 
However,  the  MCT  can  only  be  useful 
in  personnel  situations  wherein  (1)  the 
test  and  criteria  are  linearly  related 
throughout  their  full  range;  (2)  no  trait 
is  so  essential  (for  example  GCT  as  an 
index  of  reading  ability)  that  it  cannot 
be  compensated  by  high  scores  on  other 
traits ; and  (3)  the  distribution  for  each 
test  or  measure  has  a sizeable  range 
if,  indeed  it  is  to  contribute  anything  to 
the  overall  predictive  efficiency  of  the 
selection  procedure. 

Unlike  the  MCT  which  involves  a 
series  of  quite  sophisticated  statistical 
computations,  the  Multiple  Cutting- 
Score  (M  C-S)  Technique  can  easily  be 
applied  by  an  average  clerk  since  only 
averages  and  percentage  calculations 
are  ordinarily  required.  Rather  simply 
the  M C-S  technique  involves  the  de- 
termination for  each  test  in  the  battery 
a critical  score  variously  defined  as 
the  score  point  on  every  test  above 
which  the  selectees  are  accepted  and 
below  which  they  are  rejected,  or,  for 
some  selection  situations,  the  scale 
position  which  provides  maximum  dis- 
crimination between  "good"  and  "poor" 
personnel.  While  at  least  one  federal 
department,  the  United  States  Employ- 
ment Office,  has  abandoned  the  use  of 
the  MCT  methodology  in  favor  of  the 
M C-S  procedure,®  the  literature 
argues  that  both  methods  are  approxi- 
mately equivalent  in  terms  of  the  effi- 
cacy of  the  test  battery  compiled 
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for  use  in  most  personnel  selection 
programs.^ 

As  stated  earlier,  both  MCT  and 
M C-S  methods  have  been  used  for  the 
past  20  years  to  assemble  predictive 
test  score  batteries  for  a variety  of 
Naval  selection  situations  including  the 
submarine  service. -phe  sjngje 
and  multiple  test  combinations  under 
investigation  in  the  present  study  were 
based  upon  the  results  of  these  studies. 
However,  as  stated  earlier,  the  pur- 
pose of  this  study  was  to  analyze  the 
available  test  score  data  in  order  to 
identify  the  point  or  points  on  each  of 
the  test  score  distributions  which  may 
be  effeetly  employed  in  the  selection  of 
the  non-rated  segment  of  the  input  to 
Basic  Enlisted  Submarine  School. 


tical  Techniques  - Some  Assumr 


tions  and  Definitions . 


As  discussed  in  several  places  in  the 
selection  literature/0,7^/2  the  usefulness 
(validity)  of  any  selection  technique  in- 
cluding' tests  used  for  submariner  se- 
lection is  directly  proportional  to  the 
Success  Ratio  (SUR)*  in  an  unselected 
population  and  inversely  related  to  the 
Selection  Ratio  (SER)’**of  necessity  im- 
posed by  the  personnel  requirements  of 
the  situation.  Thus,  any  recommended 
methodology  for  identifying  effective 
test  score  cut-off  points  (ECOP,  Exper- 


* SUR  = for  the  present  situation , the  percentage  of  an 
unscreened  population  of  enlisted  submariner  candi- 
dates for  Submarine  School  (SubSchl)  who  succeed 
i.e.  graduate  from  SubSchl,  qualify  S/S  etc. 

** SER  = Selection  Ratio  is  the  ratio  of  number  of 
selectees  required  to  the  number  available  (usually 
presented  in  percentage  terms/.  In  a sense,  this  in- 
dex is  analogous  to  the  econometric  demand/supply 
ratio. 


imental  Cut-off  Points)  should  somehow 
take  into  account  the  prevailing  SUR/ 
SER  situation.  Accordingly,  the  modi- 
fied cutting-score  technique  used  in  this 
study  includes  an  estimate  of  the  seri- 
ousness for  the  selection  program  of 
both  types  of  incorrect  decisions  (ICD) 
i.e. , the  percent  of  false  positives  (FP) 
and  false  negatives  (FN)  for  each 
ECOP.  Since  the  notion  of  FP  and  FN 
are  central  to  the  analytical  approaches 
used  in  this  study,  a simplified  descrip- 
tion of  the  methodology  of  choice  fol- 
lows (see  also  Appendix  A) . * 

It  is  seen  in  Fig.  1 that  FP  (Quadrant 
A)  represents  those  candidates  re- 
jected by  ECOPj  but  who  would  have 
graduated  from  Submarine  School . 

* Others  have  previously  used  essentially  the  same 
reasoning  as  a means  of  determining  the  effective- 
ness of  a given  personnel  selection  system.  1 1 >20 


Conversely,  FN  (D)  are  those  men  ac- 
cepted by  ECOPj  but,  in  fact,  "drop" 
out  (fail)  SubSchl . FN  + FP  = I.C.D. 
(incorrect  decisions),  Quadrant  "B"are 
those  accepted  by  ECOPj  and  do  in  fact 
graduate  summed  with  those  rejected  by 
the  same  trial  cut-off  who  fail.  Thus 
B+C  = C.D.  (correct  decisions)  and 
A+B+C+D  = N = I.C.D.  + C.D.  = total 
personnel  decisions  to  be  made  by  the 
selection  process. 

As  implied  earlier,  the  most  mean- 
ingful ECOP  on  the  predictor (s)  score 
distribution  must  of  necessity  take  into 
consideration  the  requisite  SER  at  the 
time  the  selection  decisions  are  to  be 
made.  Thus,  if  the  number  of  optimal 
quality  men  is  in  short  supply  and  if  the 
numbers  and  quality  mixes  of  men  re- 
quired is  accelerating  (the  present  situ- 
ation in  the  submarine  service) , then 
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SER  must  be  as  near  100%  as  possible. 
Stated  simply,  if  SER  approaches  100%, 
an  ECOP  should  be  low  enough  (ECOPjj 
in  Fig.  1)  to  reject  as  few  of  the  input 
as  possible.  In  particular,  FP  should 
approach  "O"  i.e. , ECOPji  cannot 
afford  to  reject  many  (ideally,  any) 
candidates  who  would  succeed.  Con- 
versely, in  circumstances  wherein  the 
supply  of  quality  candidates  is  high  in 
proportion  to  the  personnel  require- 
ments i.e. , SER  is  low,  the  ECOP  can 
be  raised  up  the  score  scale  to,  for 
example,  ECOPjjj  (Fig.  1)  with  little 
concern  about  FP*,  but  with  much  con- 
cern about  reducing  FN*  * particularly 
if  the  training  costs  are  exhorbitant, 
for  example,  nuclear  power  submar- 
iner training. 

With  the  foregoing  selection  con- 
cepts in  mind,  the  procedure  for  pin- 
pointing the  most  efficacious  ECOP  for 
each  selection  test  or  combination  of 
tests  shown  by  earlier  studies  to  have 
significant  predictive  validity  9,10 was 
simply  to  plot  the  FN  and  FP  for  as- 
cending score  intervals.  Several  other 
indices  were  also  calculated  and  plotted 
for  each  ECOP  or  score  interval . 

These  are  defined  below: 

CD  - Correct  Decisions,  i.e., 
the  percentage  of  total 
accepted/rejected  correct 
decisions  for  each  ECOP. 

In  Fig.  1 C.D.  = [B+C/N] 

100. 

* This  is  so,  however,  only  if  the  resource  cost  (person- 
nel and  test  materials)  is  low.  Otherwise,  the  dollar 
cost  to  fail  a candidate  may  be  prohibitive.  M 

**  Another  way  of  expressing  this  selection  objective  is 
to  maximize  SUR  (Success  Ratio)  represented  by 
quadrant  “B” in  Fig.  I since  SUR  - (100— FN). 


COPY  - Coefficient  of  Practical 

Validity,  that  is  the  differ- 
ence between  correct  deci- 
sions (CD)  and  I. C.D. 
(Incorrect  Decisions),  cal- 
culated as  [B+C]  - [A+Dj/N 
changed  to  percent.  High 
positive  COPV  indices  cor- 
relate with  the  overall  effi- 
ciency of  the  selection  pro- 
gram. Conversely,  a 
negative  COPV  suggests  an 
inefficient  system. 

RL  - Relative  Losses,  that  is, 
the  difference  between  the 
proportion  of  the  men  who 
would  fail  that  would  be  re- 
jected by  the  ECOP  and  the 
proportion  of  men  who 
would  succeed  also  lost  by 
the  ECOP.  R.L.=  [C/C+D] 

- [A/A+B]  • 100*. 

One  necessary  assumption  made  in 
implementing  this  rather  simple  cut-off 
score  technique  was  that  there  were  no 
rigidly  applied  cut-off  scores  in  effect 
at  the  time  these  data  were  collected. 
This  assumption  is  necessary  since  the 
question  posed  by  this  research  viz . , 
can  a meaningful  ECOP  be  identified  for 
single  or  composite  score  distributions, 
would  be  "begged"  if  a cut-off  point 
were  in  fact  being  applied  during  the 
time  frame*  * during  which  the  data  used 
in  this  study  had  accumulated.  Thus,  in 
spite  of  the  fact  that  theoretical  BTB 

* The  reader  is  referred  to  the  dictionary  of  terms  for 
expanded  definitions  of  these  and  other  concepts 
involved  in  this  methodological  approach  (app.  A). 
**  The  data  for  this  study  were  based  upon  the  non- 
rated  input  to  Basic  Enlisted  Submarine  School 
during  the  calendar  year  1973.  The  total  input  was 
2100  of  which  the  non-rated  segment  (N=990) 
amounted  to  47%. 
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(Basic  Test  Battery)  eligibility  stand- 
ards for  submariner  candidates  have 
been  in  existence  for  more  than  a 
decade,  for  example,  GCT+Ari  = 100 
(6  point  waiver),  GCT+Mech  = 100  and/ 
or  GCT+Mech+SP  = 150  (9-point  waiv- 
er), it  was  assumed  that  because  of  the 
elevated  selection  ratio  necessitated  by 
the  ever  accelerating  requirements  for 
quality  personnel  in  short  supply,  that 
no  strict  cut-off  test  scores  were  being 
adhered  to  in  1973. 

Subjects. 

The  initial  sample  of  SR's,  SA's 
(FA's)  and  SN's  entering  Submarine 
School  between  Jan.  1,  1973  and 
Dec.  31,  1973  numbered  990.  However, 
35  of  these  men  had  been  set  back  from 
their  initially  assigned  Submarine 
School  classes  as  medical  or  other 
"holds".  Since  their  names  may  have 
appeared  on  more  than  one  class  ros- 
ter, they  were  eliminated,  the  remain- 
ing sample  being  955.  The  median  age 
was  18.4  years  with  a range  of  17  to 
30  years.  Sixty-four  percent  of  the 
sample  were  SR's,  20  percent  were 
SA/FA's  and  the  remainder  SN's. 

Data  Processing  Techniques. 

Selected  items  of  biographical  in- 
formation, the  BTB  score  combinations, 
and  Submarine  School  criteria* **  were 
coded  and  IBM  key  punched  by  the  ADP 
Branch  of  the  Laboratory*  *.  Besides 

* A 20-30  column  punched  card  field  was  left  blank  so 
that  a battery  of  personality  test  scores  could  be 
punched  to  be  analyzed  at  a later  date. 

** Appreciation  is  extended  to  Mrs.  Betty  Megos,  Head 
of  the  ADP  Branch  and  her  staff  for  punching  and 
verifying  the  cards  and  for  performing  the  numerous 
card  sorting  operations  required  to  produce  the  ac- 
cumulative percentage  distributions  basic  to  the 
study. 


the  key  punch  and  verifier , the  only 
ADP  hardware  used  in  this  study  was 
the  IBM  026  sorter  with  pocket  counter 
attachment  by  means  of  which  the  rank- 
ordered  frequency  distributions  for 
each  of  the  test  score  combinations 
were  obtained.  The  accumulative  per- 
centage distributions  and  the  five  score 
cut-off  indices  were  derived  for  each 
score  combination  (See  Appendix  A) 
from  these  rank  order  distributions  by 
means  of  a desk  calculator . The  total 
computation  time  for  each  test  score 
or  score  combination  was  approxi- 
mately three  man-hours. 

RESULTS 

As  implied  in  the  introductory  sec- 
tion of  this  paper,  the  present  study  is 
an  extension  or  updating  of  past  re- 
search aimed  at  delineating  the  most 
effective  predictors  of  Submarine  School 
success.  Based  upon  research  extend- 
ing over  more  than  two  decades  9,10,15,16 
the  relative  predictive  effectiveness  of 
all  possible  combinations  of  BTB  scores 
and  other  measures  has  been  repeatedly 
calculated  on  independent  samples  of 
enlisted  candidates  for  the  submarine 
service.  The  present  study  extends 
these  earlier  validation  studies  in  two 
ways:  First,  past  studies  have  focussed 
only  upon  samples  of  total  input,  both 
rated  and  non-rated,  to  the  Submarine 
School.  In  contrast,  the  population 
sample  for  the  present  study  included 
only  non-rated  personnel  who  had  vol- 
unteered for  submarine  training.  Sec- 
ondly, based  upon  the  assumption  that 
optimal  test  score  combinations  had 
been  delineated  by  past  studies  over  the 
years,  the  present  study  submits  these 
test  score  distributions  to  an  analysis 
designed  to  identify  for  each  distribution 
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a cut-off  point  which,  if  applied,  will 
maximize  the  efficiency  of  the  submar- 
iner selection  system  for  the  SER  and 
SUR  (See  Appendix  A)  existing  at  a 
given  time. 

The  Search  for  Moderator  Variables. 

Previous  studies  designed  to  exam- 
ine the  interactional  effects  of  BTB 
scores  and  personality  test  scores  have 
demonstrated  the  necessity  of  exploiting 
the  occurrence  of  moderator  factors  in 
a personnel  selection  situation  (See 
Appendix  A for  clarification  of  the 
moderator  concept). 


Educational  Achievement  and  SubSchool 
Attrition.  The  significant  moderator 
effect  of  EA  (Educational  Achievement) 
for  prediction  of  Submarine  School  Per- 
formance has  been  known  for  almost 
twenty  years  (See  Table  3 in  reference 
No.  15).  Thus,  in  the  mid-fifties  it  was 
known,  for  example,  that  submariner 
candidates  who  had  graduated  from  high 
school  had  2-1/2  times  the  probability 
to  graduate  from  SubSchl  as  did  high 
school  dropouts  (op.  cit.,  p.  4).  The 
relationship  between  EA  and  SubSchl 
attrition  is  depicted  for  the  sample  of 
955  non-rated  submariner  candidates  in 
Table  I. 


Table  I.  Distributions  of  Education  Achievement  of  Non-rated 
Enlisted  Sample  (N=955) 


Educational  Achievement  (EA) 

SubSchool 

Status 

' 

9 years  or 
less 

10  - 11  years 

12  years 

More  than 
12  years 

i 

% 

f 

% 

f 

% 

f_ 

% 

Drops 

45 

53.6 

151 

47.9 

110 

22.0 

5 

8.9 

Graduates 

39 

46.4 

164 

52.1 

390 

78.0 

51 

91.1 

Totals 

84 

100 

315 

100 

500 

100 

56 

100 

% of  Total  in 
each  EA  Group 

8.8 

33.0 

52.3 

5.9 

With  df=  3,  p of  X2  = <0.01 
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At  the  outset,  clarification  of  the 
term  Submarine  School  (SubSchl)  "drop" 
as  used  in  Table  I is  required.  Eighty- 
two  percent  of  the  311  "drops"  were 
labelled  "academic"  indicating  that  the 
major  cause  of  the  failure  had  to  do 
with  the  inability  to  acquire  sufficient 
knowledge  and  skills  to  meet  the  "pass- 
ing" standards  required  by  the  SubSchl. 
The  remaining  56  men  (18%)  were  the 
so-called  "administrative"  drops",  men 
disqualified  for  insufficient  obligated 
time,  and  for  disciplinary,  psychologi- 
cal, and  other  reasons.' 

Overall,  the  attrition  rate  (AR)  was 
33%  in  this  sample.  It  is  immediately 
apparent  from  the  data  in  Table  I that 
the  "betting  odds"  are  49%  to  fail  in 
SubSchl  if  a candidate's  EA  is  less  than 
12  years.  This  compares  to  a probabil- 
ity to  fail  of  21%  if  he  has  completed 
high  school  (EA>12).**  Stated  another 
way,  63%  of  the  SubSchl  drops  in  this 
sample  were  also  high  school  dropouts, 
while  only  31%  of  the  SubSchl  graduates 
had  an  EA  less  than  12.  Finally,  there 
is  no  statistically  significant  (5%  con- 
fidence level,  by  Chi2)  in  the  odds  to 
fail  in  SubSchl  of  those  with  EA  = 9 and 
those  with  EA=10-11  years,  the  odds 
being  about  50/50.***  Thus  from 
Table  I,  if  an  EA  of  12  or  more  were 
required  for  qualification  of  non-rated 
men  for  submarine  training,  41.8  of  the 

* The  relatively  few  candidates  disqualified  for  the  sub- 
marine service  for  medical  reasons  prior  to  the  onset 
of  the  basic  submarine  course  were  not  included  in 
the  “drop” group. 

**The  1.5%  (14  men)  who  had  obtained  a high  school 
diploma  by  means  of  the  G.E.D.  equivalency  exam- 
ination were  included  in  the  high  school  drop 
(EA  < 12)  group. 

***A  Sample  of  SubSchl  enlisted  men  obtained  in  1956 
showed  exactly  the  same  odds  for  high  school  drop- 
outs to  graduate  from  SubSchl,  namely,  50%.  15 


non-rated  volunteers  in  the  present 
sample  (N=  955)  would  have  been  re- 
jected. However,  of  those  high  school 
graduates  accepted  79%  (556  men) 
would  graduate.  Perhaps  noteworthy  is 
the  fact  that  this  hypothetical  SubSchl 
output  of  441  graduates  is  31%  less  than 
the  output  of  644  (67%  of  total)  in  the 
present  sample.  The  question  becomes 
however,  if  the  requirement  of  EA=12 
be  imposed  upon  submariner  candidates, 
can  the  existing  personnel  situation 
withstand  the  51%  false  positive  (FP, 
see  Appendix  A)  loss. 

For  purposes  of  comparison  of  the 
effectiveness  of  the  various  ECOP's 
(Experimental  Cut-off  Points)  to  be 
identified  for  the  test  score  distribu- 
tions later  on  in  this  paper,  the  same 
indices  were  calculated  for  a hypotheti- 
cal ECOP/EA4  12  the  plausibility  of  which 
is  suggested  by  the  data  in  Table  I. 
Following  the  rationale  expressed  in  the 
discussion  of  Fig.  1,  the  following  in- 
dices* were  calculated  for  ECOP/ 

EA<  12:  CD=66.7%,  COPV  = 33.4%, 
RL-31%,  FP=  51%,  and  FN=21%.  Thus 
this  simple  procedure  would  hypotheti- 
cally result  in  a rather  efficient  selec- 
tion system  resulting  in  a SUR  (Success 
Ratio)  of  100-FN=79%.  However,  the 
price  paid  for  imposition  of  this  proce- 
dure is  reflected  by  the  statistic  RL= 
31%  resulting  from,  rejecting  63%  of  the 
potential  "Drops"  but  at  the  same  time 
losing  32%  of  those  men  who  would 
graduate  from  Submarine  School.  It  re- 
mains to  be  proven  whether  any  proce- 
dure involving  test  score  ECOP's  will 
significantly  improve  this  simple  EA 
cut-off  procedure.  At  any  rate,  it  is 

* See  Appendix  A and  the  discussion  of  Fig.  1 for 
an  explanation  of  these  indices  of  selection 
efficiency. 
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obvious  from  the  data  in  Table  I and  the 
above  discussion  that  submariner  can- 
didates who  are  and  are  not  high  school 
graduates  show  significant  differences 
in  terms  of  the  present  criterion  of 
concern,  drop/graduate  from  SubSchl. 
Thus , EA  appears  without  question  to 
be  a significant  moderator  variable, 
necessitating  the  identification  of  effec- 
tive ECOP's  for  each  of  the  selection 


tests  separately  for  the  segments  of  the 
non-rated  population  with  and  without  a 
high  school  diploma. 

Age  and  SubSchl  Attrition.  Another 
variable  of  concern  is  the  relationship 
of  age  of  the  non-rated  candidate  to  Sub- 
Schl attrition.  Table  n presents  these 
data  in  crossbreak  form.* 


Table  II.  Age  Distributions  for  Submarine  School 
Graduate  and  Drop  Sample  (N=  955) 


SubSchool 


AGE 


Status 

17 

18-19 

20-22 

l 

23+ 

1 

% 

i 

% 

f 

% 

_f 

% 

Drops 

99 

49.3 

164 

29.3 

44 

26.2 

4 

15.4 

Graduates 

102 

50.7 

396 

70.7 

124 

73.8 

22 

84.6 

Totals 

201 

100 

560 

100 

168 

100 

26 

100 

% of  Total  in 
each  age  group 

21.0 

58.6 

17.6 

2.7 

With  df=2,  p of  X2=<0.01 


It  is  immediately  obvious  that  the 
21%  of  the  non-rated  input  to  Submarine 
School  under  17  years  of  age  are  a high 
risk  group  (AR=49) , which  is  signifi- 
cantly different  from  the  18  and  older 
candidates  for  which  an  AR=28%  is 
found.  Thus,  imposing  a minimum  age 
requirement  of  18  for  non-rated  volun- 
teers would  bring  about  the  elimination 
of  a poor  risk  group  resulting  in  the 
loss  of  32%  of  those  who  would  eventu- 


ally drop  but  at  a "cost"  of  16%  of  those 
who  would  graduate  thus  yielding  an  EL 
(Relative  Loss)  of  16%  (See  Appendix  A). 
In  light  of  the  discussion  of  the  SER 
concept  in  the  methods  and  procedure 
section,  application  of  this  age  cut-off 


A crossbreak  or  contingency  table  is  a method  of 
presenting  numerical  data,  usually  in  frequency  or 
percentage  form,  in  which  variables  or  factors  are 
juxtaposed  in  such  a manner  that  their  interrela- 
tionships can  be  examined. 
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point  can  be  recommended  only  if  an  high  school  graduates  is  18  years,  the 
SER  of  79%  or  less  for  the  non-rated  likelihood  that  the  SubSchl  attrition 

segment  of  the  input  to  SubSchl  is  data  in  Tables  I and  II  actually  "tell  the 

feasible  (79%=  the  complement  of  the  same  story",  namely  that  the  high 

21%  loss  of  the  17  year  olds  caused  by  risk  candidate  group  whose  EA  is  less 

the  ECOP  being  set  at  age  17).  than  12  is  mainly  composed  of  17  year 

olds,  also  showing  an  elevated  AR. 

Age  and  Educational  Achievement  Table  III  presents  this  three-way 
Interaction.  Since  the  mean  age  of  crossbreak. 


Table  HI.  Age  and  Educational  Achievement  Interactional 
Relationship  with  Attrition  Rate  (AR)  in 
Submarine  School  (N  = 950c) 


Educational 
Achievement  (EA) 

Age 

17 

Dropsa  Gradsa 

20+ 

Drops  Grads 

% 

in  each 
EA  Group 

9 years  or  less 

25  25 

13  11 

6 3 

8.7 

AR=50% 

AR=54% 

AR=66% 

10-11  years 

66  67 

73  82 

12  15 

33.0 

AR=50% 

AR=47% 

AR=44% 

12  years  or  more 

8 10 

73  303 

30  128 

58.2 

AR=44% 

AR=20% 

AR=19% 

Totals 

99  102 

159  396 

48  146 

% Total  in  each 
Age  Group 

20.1 

58.4 

21.5 

With  df=2,  p of  X2 

_ _ b 

n.s. 

.001 

<.001 

cl 

Submarine  School  Drops  and  Graduates 
to 

n.s.  - nonsignificant,  i.e.,  null  probability  > 5% 
Incomplete  data  on  5 cases. 
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As  expected,  92%  of  the  17  year  old 
submariner  volunteers  are  high  school 
dropouts  who  have  about  50%  probability 
to  graduate  from  SubSchl  irrespective 
of  EA  {See  column  1 in  Table  HI).* ** 
Further,  ah  examination  of  the  AH's  for 
age  groups  "18-19"  and  "20+"  (columns 
2 and  3)  reveals  that  an  AR  less  than 
20%  would  result  from  accepting  non- 
rated  volunteers  who  are  at  least  18  and 
are  high  school  graduates. 

The  question  as  to  whether  the  im- 
position of  an  age/EA  interactional  cut- 
off would  significantly  improve  the 
selection  situation  for  the  non-rated 
input  to  SubSchl  was  raised.  Accord- 
ingly, the  indices  calculated  from 
Table  in  for  an  EC  OP  EA<12  & Age<GL8 
were:  COPV=33.5%,  CD=  67%,  FP  = 
51%,  FN=  19.3%  and  RL=  33.2  (drop 
loss  66.3,  graduate  loss  33.1).  Thus, 
including  the  age  17  criterion  with  the 
EA<  12  cut-off  based  upon  Table  I would 
result  in  only  a very  slight  improve- 
ment, namely  2%  increase  in  RL  and  a 
drop  of  about  the  same  proportion  in 
FN.  However,  as  before,  the  permis- 
sible SER  for  the  non-rated  component 
of  the  submariner  volunteer  pool  must 
be  of  the  order  of  50%*  for  this  age/ 
EA  cut-off  group  to  be  practical. 


cade  and  E A Interaction.  Since 


paygrade  is  obviously  correlated  with 


* Thirty-two  percent  of  the  18-19  y/o  and  37%  of  the 
20+ age  were  high  school  dropouts  in  the  present 
sample  of  non-rated  SubSchl  candidates. 

**  This  statistic  was  calculated  by  first  summing  the 
total  losses  of  men  resulting  from  this  EA  j Age  cut- 
offi.e.,A+C+D  in  Fig.  1.  The  complement  of  this 
sum  changed  to  % of  total  is  the  percentage  of  this 
group  who  would  finally  graduate  from  Submarine 
School. 


time-on-active-duty,  the  possibility 
that  the  more  adequately  indoctrinated 
SN's  and  SA's  would  be  better  risks  for 
SubSchl  was  examined  next.  Table  IV 
contains  data  bearing  on  this  point  and 
analyzed  separately  for  the  high  school 
graduate  and  high  school  dropout  com- 
ponent of  the  sample. 

Again  as  expected,  E-l's  are  a high 
risk  group  (AR=39%  overall)  but  much 
more  for  high  school  dropouts  (AR= 
48.7%)  than  for  high  school  graduates 
(AR  = 29. 4%) . With  increase  in  pay- 
grade  the  AR  situation  improves  signi- 
ficantly but  only  for  high  school  gradu- 
ates again  providing  support  for  the 
treatment  of  the  EA  as  a moderator 
variable  in  search  for  effective  ECOP's 
for  the  test  score  distributions  to 
follow. 


Gross  Descriptive  Statistics  for  each 


Test  or  Test  Score  Combination. 


As  stated  earlier,  the  purpose  of  this 
study  was  to  search  the  distributions  of 
selected  tests  and  test  score  combina- 
tions obtained  from  a sample  of  non- 
rated  submariner  volunteers  to  ascer- 
tain whether  meaningful  cut-off  points 
could  be  identified.  Before  initi- 
ating the  cut-off  score  analysis 
however,  it  was  necessary  to  com- 
pare the  score  distributions  for  the 
tests  and  test  combinations  taking 
into  account  the  major  moderator 
variable.  Educational  Achievement 
(EA).  Table  V contains  gross  de- 
scriptive statistics  estimated  from 
the  accumulative  frequency  and  per- 
centage distributions  derived  by  a 
card  sorter  and  desk  calculator 
for  each  test  score  or  combina- 
tion. 
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Attrition  rate  within  EA  group  across  all  paygrades. 


Table  V.  Gross  Descriptive  Statistics  for  the  Distributions 
of  Each  Test  or  Test  Score  Combination 


E.A. 

Test(s)a  Grouping 

S/S 

Status 

Sample 

size 

Median 
(50  p.c.) 

16  p.c. 

84  p.c. 

(J 

Approximate 

S.D. 

Symmetry^ 

S/S  Grads 

203 

53.3 

46.4 

59.9 

6.7 

Symmetrical 

<12 

S/S  Drops 

195 

50.3 

45.7 

56.9 

5.6 

Slight  + 

GCT 

S/S  Grads 

441 

57.6 

49.8 

65.3 

7.8 

Symmetrical 

>12 

S/S  Drops 

116 

50.9 

46.1 

56.8 

5.3 

Slight  + 

S/S  Grads 

203 

103.4 

96.0 

113.3 

8.6 

Slight  + 

<12 

S/S  Drops 

163 

99.7 

91.9 

105.7 

6.9 

Slight  - 

GCT  + MECH 

S/S  Grads 

441 

108.4 

98.4 

121.2 

11.4 

Slight  + 

>12 

. 

S/S  Drops 

92 

97.5 

90.0 

106.5 

8.2 

Slight  + 

S/S  Grads 

182 

154.1 

142.7 

171.5 

14.4 

Mod.  + 

<12 

• 

S/S  Drops 

149 

149.2 

140.9 

161.9 

10.5 

Mod.  + 

GCT  + MECH  + SP 

S/S  Grads 

424 

162.3 

146.8 

180.1 

16.6 

Symmetrical 

>12 

S/S  Drops 

102 

146.1 

136.5 

163.8 

13.6 

Sharp  + 

S/S  Grads 

182 

150.2 

138.9 

165.7 

13.4 

Slight  + 

<12 

S/S  Drops 

150 

147.1 

136.1 

158.3 

11.1 

Symmetrical 

ARI  + MECH  + SP 

S/S  Grads 

423 

158.5 

143.7 

175.1 

15.6 

Symmetrical 

>12 

S/S  Drops 

103 

145.  6 

131.9 

155.4 

11.7 

Slight  - 

S/S  Grads 

605 

53.1 

44.5 

60.2 

7.8 

Symmetrical 

SP 

S/S  Drops 

210 

48.9 

42.9 

55.4 

6.2 

Symmetrical 

aGCT  = General  Classification  Test;  MECH  = Mechanical  Aptitude  Test;  SP  = Shop  Practice 
Test. 

^E. A.  = Educational  Achievement 


Q 

S.D.  = Standard  Deviation  approximated  from  mean  score  range  between  84  p.c.  and 
16p.c. 
cl 

Skewness  estimated  from  relative  difference  between  the  score  distance  from  50  p.c.  to 
16  p.  c.  and  from  50  p.  c.  to  84  p.c. 
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At  the  outset,  it  is  obvious  that  the 
differences  between  the  EA  groupings 
(high  school  drops  versus  high  school 
graduates)4"  in  terms  of  the  score  dis- 
tributions are  in  most  cases  sizeable, 
thus  providing  further  support  for 
treating  EA  as  a moderator  variable 
throughout  the  score  cutoff  analysis  to 
follow.  Another  related  observation  of 
some  considerable  significance,  is  that 
the  score  distributions  for  the  EA<12 
groups  are  very  similar  for  Submarine 
School  Drops  as  compared  to  graduates. 
This  was  not  true,  however,  for  the 
score  distributions  obtained  from  the 
high  school  graduate  segment,  as  large 
distribution  differences  between  Sub- 
Sehl  graduates  and  drops  appeared  for 
each  test  combination  for  this  EA  sub- 
group. The  upshot  of  this  finding, apart 
from  providing  further  justification  for 
the  EA  moderator,  is  that  the  high  school 
dropout  component  of  the  SubSchl  input 
is  rather  homogeneous  insofar  as  the 
presently  used  test  battery  is  con- 
cerned . It  follows  therefore  that  the 
identification  of  optimal  cut-off  scores 
should  be  much  more  difficult  for  the 
high  school  dropouts  than  for  the  high 
school  graduate  subsample. 

Optimal  Experimental  Test  Score 
Cut-off  Points. 

As  stated  in  the  procedural  section, 
the  data  analysis  for  each  of  the  dis- 
tributions of  test  or  test  score  combin- 
ations was  identical.  Defined  in  Ap- 
pendix A,  five  interrelated  indices  (CD, 
FN,  FP,  COPV  and  RL)  were  derived 
for  each  ECOP,  which  were  in  all 

* Preliminary  examination  of  the  distributions  of  the 
SP  (Shop  Practice)  test  scores  indicated  that  parti- 
tioning the  data  for  different  EA  levels  was  not 
necessary , 


cases  separated  by  a 5-point  score 
interval. 

GCT.  The  line  graphs*  for  each  of 
these  5 indices  are  presented  for  GCT 
distributions  obtained  for  the  high 
school  drop  group  in  Fig.  2A  and  for 
the  high  school  graduate  group  in  Fig. 

2B. 

Looking  first  at  Fig.  2A,  GCT=  50 
is  sharply  delineated  as  the  ECOP 
where  the  screening  system  is  most 
efficient  as  indicated  by  a maximum 
COPV  of  20%.  But,  the  personnel 
losses  would  be  high  at  that  point, 
namely,  43%  (A  +C/N  in  Fig.  1). 

Further,  the  FP  index  is  40%  at  GCT= 

50  i.  e. , 40%  of  those  rejected  by  the 
ECOP=  50  would  have  been  graduated 
had  they  have  been  accepted.  Equally 
troublesome  is  the  FN=  40%  at  ECOP  = 

50  i.e. , 40%  of  those  accepted  (B  + D 
in  Fig.  1)  will  drop  in  SubSchl,  a sta- 
tistic that  is  slightly  lower  than  the  ex- 
pected attrition  of  high  school  dropout 
candidates  for  the  submarine  service 
irrespective  of  their  GCT  scores.  In 
sum,  with  both  FP  and  FN  about  40% 
the  imposition  of  an  ECOP=  50  would 
result  in  an  SUR  of  about  60%.  In  other 
words,  the  attrition  rate  for  the  non- 
rated,  EA<12,  group  would  be  reduced 
from  the  existing  49%  (Table  I)  to  40% 
with  the  imposition  of  ECOP=  GCT/5of  * 

* The  reader  should  realize  that  line  graphs  are  used 
merely  for  convenience  since  the  abscissae  of  all  the 
figures  to  follow  are  discrete  rather  than  continuous 
scales  consisting  of  ascending  ECOP’s  separated  by 
5-point  score  intervals.  The  interconnecting  data 
points  were  not  calculated. 

**A  caveat  is  necessary  at  this  point,  namely,  that  all  of 
the  statistical  indices  used  in  this  study  (COPV,  FN, 
RL,  etc.),  like  all  statistics,  have  sampling  errors. 
Accordingly,  computed  on  a different  subject  sample, 
these  indices  may  be  quite  different,  resulting  in  dif- 
ferent ECOP’s,  each  with  correlated  selection  indices. 
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Score  Intervals 

Fie.  2 A.  Experimental  cut-off  points  for  GCT 

(SS  Grads.,  N=203;  S/S  Drops  N=1 95; 

E.A.  < 12) 

Turning  to  the  line  graphs  in  Fig. 

2B,  as  might  be  expected  from  the  ele- 
vated GCT  distributions  for  high  school 
graduates  (Table  V) , the  present  selec- 
tion system  for  this  higher  EA  group  is 
slightly  more  efficient  at  ECOP's  GCT/ 
40-55.  This  is  so  because  the  AH  in 
this  group  overall  is  only  21%  (Table  I) 
and  decreases  to  8%  at  GCT=  55.  The 
fact  that  FP  "climbs"  from  62%  at 
ECOP  GCT/45  to  72%  at  GCT=  60  would 
seem  to  argue  against  the  application 
of  any  ECOP  based  on  GCT  scores  for 
this  group , especially  when  a high  SER 
is  necessitated  by  the  extant  personnel 
situation. 

One  final  comment  regarding  the 
possibility  of  locating  a minimal  ECOP 
on  the  GCT  scale  below  which  a sub- 
mariner candidate  would  absolutely  be 
disqualified  presumably  on  the  basis 


LEGEND: 


Score  Interior* 

Fig.  2B.  Experimental  cut-off  points  for  GCT 
(SS  Grads.,  N=441;  S/S  Drops,  N=116; 

e.a.  ;> 12 ) 

of  illiteracy  or  limited  literacy.  In 
terms  of  the  indices  of  selection  effi- 
ciency used  in  this  study  this  hypo- 
thetical point  on  the  GCT  distribution 
would  coincide  with  the  ECOP  at  which 
FP=  0 i.e. , all  candidates  rejected 
would  have  failed.  Looking  at  Fig.  2B 
first,  with  FP=50  at  ECOP  GCT/40, 
it  is  obvious  that  such  a "bottom^out" 
score  is  non-existent  for  the  EA=  12 
input  to  SubSchl.  However,  this  may 
not  be  true  for  the  EAC12  subgroup. 
Accordingly,  if  we  extrapolate  the 
FP  line  in  Fig.  2A  from  GCT/45 
through  GCT/40  backward  it  crosses 
the  GCT  score  scale  exactly  at 
GCT/33.  Keeping  in  mind  that  only 
2. 5%  of  the  present  sample  (10  men) 
had  GCT  scores  below  40,  nonetheless, 
hypothetically  at  least,  this  minimal 
score  may  be  at  or  near  GCT=33  for 
similar  non-rated  samples. 
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GCT  + MECH.*  The  same  type  of 
analysis  was  completed  for  the  com- 
bined T-Score  distributions  for  the 
combined  GCT  and  Mech  scores  ob- 
tained from  the  present  sample  of  non- 
rated  submariner  candidates,  again 
treating  the  data  separately  for  high 
school  dropouts  and  for  high  school 
graduates.  Figs.  3 A and  3B  contain 
the  results  of  this  analysis . 

For  those  men  with  less  than  12 
years  formal  education,  the  plots  in 
Fig.  3 A suggest  that  an  ECOP = GCT  + 
MECH/100  is  more  efficient  than 
ECOP  = GCT/50  for  the  same  group 
(COPV  = 25%  as  compared  to  19%).  Yet 
the  AR  for  this  ECOP  would  still  be 
about  33%  at  that  point.  Moreover,  48% 

* ARI  + MECH  score  distributions  were  not  analyzed 
on  the  basis  that  the  correlation  of  this  composite 
score  with  SubSchl  criteria  has  been  shownl  0 to  be 
approximately  equivalent  with  the  same  statistic 
calculated  for  GCT  + MECH  (Figs.  3A  and  3B)  and 
therefore  the  test  score  cut-off  potentiality  tvould 
similarly  be  expected  to  be  comparable. 


60  95  90  95  100  105  1 10  115  120  125  130  05  140 

Score  Intervols 


Fig.  3A.  Experimental  cut-off  points  GCT  +MECH 

(SubSchool  Drops  N=163,  Grads.  N=203, 
E.A.  < 12) 


of  the  total  input  in  this  group  would  be 
rejected  by  this  ECOP  (A  + C in  Fig.  1) 
of  which  43%  (about  21%  of  the  total 
input)  would  have  graduated  had  they 
been  accepted. 

A slightly  more  effective  ECOP  for 
the  GCT  + Mech  distributions  for  non- 
high  school  graduates  can  be  pinpointed 
in  Fig.  3A  viz. , ECOP  GCT  + MECH/ 
95.  At  this  point,  AR  (FN)  would  in- 
crease to  39%  but  9%  fewer  "good"  can- 
didates would  be  rejected  as  indicated 
by  the  drop  of  9%  in  FP  between  these 
two  ECOP's. 

As  for  a possible  ECOP  for  the  GCT 
+ MECH  distributions  for  the  high 
school  graduate  component  of  the  non- 
rated  input  to  the  submarine,  the  data 
in  Fig.  3B,  suggest  that  an  AR  of  less 
than  20%  would  result  from  accepting 
all  of  this  group  irrespective  of  their 
combined  GCT  + MECH  scores. 


60  65  90  95  100  105  110  115  (20 


Score  lntftrvols 

Fig.  3B.  Experimental  cut-off  points  GCT  + MECH 

(S/S  Grads.,  N=441,  Drops  N-92,  EA  > 12) 
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However,  an  ECOP  GCT  + Mech/90 
would  minimize  FP*,  which  may  be  de- 
sirable when  SER  must  be  high. 

GCT  + MECH  + SP . The  line  graphs 
for  the  distributions  of  summed  stand- 
ard scores  for  GCT,  MECH  and  SP  are 
presented  in  Figs.  4A  and  4B. 

Whereas  it  cannot  be  directly  esti- 
mated from  Figs.  4A  and  4B,  it  should 
be  noted  that  for  the  EAC12  group  (Fig. 
4A),  about  17%  (N=331)  were  below  the 
hypothetical  cut-off  of  141  (150  less  the 
allowable  9-point  waiver) . The  equiva- 
lent proportion  for  the  EA>12  group 
(Fig.  4B)  was  of  the  order  of  7%. 
Overall,  the  selection  efficiency  of  this 
score  combination  is  not  high  as  esti- 
mated by  COPY  of  18%  as  a maximum 
at  E COP/150  with  a coincident  CD  only 

* The  number  of  cases  upon  which  this  FP  calculation 
was  based  was  small,  i.e.,  N~32  or  6%  of  the  total. 


of  59%  (chance  probability  = 50%).  For 
the  EA<12  group,  this  combination  ap- 
pears approximately  equivalent  to  GCT 
alone  (Fig.  2A);  however,  with  AR  36- 
44%  none  of  the  test  scores  singly  or  in 
combination  appear  to  offer  much  prom- 
ise as  a selection  system  for  this  EA 
group. 

An  examination  of  the  same  graphs 
obtained  from  the  higher  EA  group  (Fig. 
4B)  as  before,  suggests  that  acceptance 
of  candidates  regardless  of  their 
summed  test  would  yield  less  than  a 
20%  attrition  rate.  However,  if  a low 
SER  were  ever  feasible  (say  60%  or 
less),  judging  from  the  CD  and  COPV 
graphs,  an  ECOP  in  the  145-155  might 
be  useful  in  the  sense  that  an  AR  of 
9-13%  would  be  realized.  This  hypo- 
thetical situation  is  feasible  because 
with  a low  SER,  the  number  of  capa- 
ble men  rejected  (high  FP)  is  not  of 
great  concern.  However,  with  the  req- 
uisite Selection  Ratio  high  (90%  or  so) 


^cceno: 
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Fig.  4A  . Experimental  cut-off  points  for  GCT  + MECH 
+ SP  (SS  Grads.  N=182,  SS  Drops  N=149, 

E.A.  < 12) 


Fig.  4B.  Experimental  cut-off  points  for  GCT  + MECH 

+ SP  (SS  Grads.  N=424;  SS  Drops  N=102; 

E.A.  ^12) 
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no  GCT  + MECH  + SP  ECOP  is  indicated 
for  this  EA  group. 

ARI  + MECH  + SP.  Whereas  past 
research  has  shown  the  combination  of 
API  + MECH  and  GCT  + MECH  to  be 
approximately  equivalent  in  the  sense  of 
predictive  capability  in  terms  of  Sub- 
marine School  performance  criteria/0 
nonetheless  it  seemed  desirable  to  eval- 
uate both  combinations  with  SP.  Fig- 
ures 5A  and  5B  contain  the  graphs  for 
these  two  distributions. 

It  is  immediately  obvious  from  the 
COPV  plot  in  Fig.  5A  which  is  13%  or 
less  at  all  ECOP's,  and  from  the  CD 
line  only  slightly  above  chance  (maxi- 
mum 57%)  that  this  score  combination 
is  not  practically  useful  with  this  lower 
EA  group . This  is  certainly  true  for 
scores  up  to  145  as  more  than  40%  of 
those  selected  would  fail  (FNM0%)  and 
at  the  same  time  almost  half  of  those 


men  rejected  by  ECOP's  in  that  range 
would  have  succeeded  (FP=50%). 

As  for  the  practicality  of  any  identi- 
fiable ECOP  for  the  higher  EA  segment 
of  the  enlisted  sample,  the  graphs  in 
Fig.  5B  are  very  similar  to  those  re- 
sulting from  the  GCT/MECH/SP  analy- 
sis depicted  in  Fig.  4B  showing  fairly 
efficient  ECOP's  in  the  lower  score 
range  with  COPV  peaking  at  ECOP  135, 
dropping  rapidly  because  of  a rapidly 
rising  FP.  But  as  in  all  of  the  previous 
test  score  analyses  involving  only  the 
high  school  graduate  portion  of  the 
present  sample,  irrespective  of  a can- 
didate's combined  ARI/MECH/SP  score 
the  betting  odds  are  more  than  80%  to 
graduate  from  Submarine  School.  In- 
terestingly, if  the  personnel  require- 
ments were  such  that  only  half  of  the 
EAj?12  volunteer  group  were  needed 
(i.e.  SER=  50%),  an  ECOP  of  155  would 
result  in  a 94%  Success  Ratio,  while 


125  130  133  140  145  150  155  160  165  170  175  ISO  1 83 
Scor«  ln}«rvali 


Fig.  5A.  Experimental  cut-off  points  for  ARI  + 

MECII  + SP  (E.A.  < 12;  SubSchl  Grads. 
N=182,  SubSchl  Drops  N=I  50) 
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Fig.  5B.  Experimental  cut-off  points  for  ARI  + 
MECH  + SP  (E.A.  ^ 12;  SubSchl  Grads. 
N=423;  SubSchl  Drops  N=103) 
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rejecting  84%  of  potential  failures  at  a 
"cost"  of  rejecting  43%  of  the  potential 
"successes",  i.e.,  R.L.=  41%. 

SP.  Finally,  since  the  Shop  Practice 
(SP)  test  was  a relatively  recent  entry 
to  the  U.S.  Navy  selection  test  battery, 
the  same  analysis  was  completed  for 
the  SP  distribution,  without  this  time, 
controlling  for  individual  differences  in 
Educational  Achievement.  The  result- 
ing graphs  are  contained  in  Fig.  6. 

A comparison  of  the  descriptive  sta- 
tistics for  the  SP  scores  obtained  from 
the  Submarine  School  graduates  (N=  605) 
with  those  from  the  drop  group  (N=210)* 
indicates  rather  similar  distribution 
(Table  V),  although  the  difference  be- 
tween the  medians  was  significant  (1% 
confidence  level,  t-test).  While  the 
COPV  index  is  highest  on  the  low  score 
end  of  the  SP  score  scale,  FP  in  that 
score  range  is  60%  or  higher  making  the 

* Forty  men  in  the  sample  did  not  have  SP  scores. 


Se»r*  Intsrvols 

Fig . 6.  ■ ' Experimental  cut-off  points  for  SP 

(SS  Grads.  N=605;  S/S  Drops  N=210) 


selection  "cost"  of  losing  capable  sub- 
mariner candidates  prohibitively  high 
during  a high  SER  era.  Too,  the  non- 
feasibility of  employing  any  E COP  on  the 
SP  score  scale  is  indicated  by  the  fact 
that  CD  (Correct  Selection  Decisions) 
approaches  50%  in  the  maximum  fre- 
quency density  range,  i.e.  the  50-60 
percentile  area. 

SUMMARIZING  STATEMENTS 

Several  limitations  of  the  rather 
simple  methodology  used  in  this  study 
to  identify  possibly  useful  ECOP's  for 
several  moderator  variables  and  test 
score  combinations  have  already  been 
mentioned.  Thus,  the  question  of  the 
reliability  of  the  indices  of  selection 
efficiency  (COPV,  CD,  RL,  FN,  FP) 
was  raised  earlier.  An  extension  of 
this  sampling  problem  is  that  these 
indices,  calculated  for  ECOP’s  at  var- 
ious levels  of  the  score  scale  are  dif- 
ferentially reliable  as  a result  of 
varying  frequency  density.  An  ex- 
treme example  of  this. characteristic  of 
the  test  score  distributions  may  be  seen 
in  Fig.  6 at  ECOP  SP/35,  where  FP= 
50%  computed  on  the  basis  of  2 out  of  a 
total  of  4 cases  below  this  ECOP.  Cal- 
culations in  the  15  to  85  p.c.  ranges 
are  of  course  based  upon  much  more 
copious  subject  samples.  Another  limi- 
tation related  to  the  reliability  problem 
is  the  fluctuation  of  the  pass/fail  level 
of  success  criteria,  for  example,  diffi- 
culty level  of  achievement  tests  in  Sub- 
marine School  from  time  to  time. 

Finally,  one  criticism  often  stated  by 
the  proponents  of  the  MCT  (Multiple 
Correlation  Techniques)  applies  to  the 
several  exploratory  analyses  involving 
summed  test  scores  (in  standard  score 
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form)  each  with  unitary  "weight"  rather 
than  with  precisely  determined  Beta 
weights  capable  of  being  derived  by  the 
proper  application  of  MCT  to  this  multi- 
ple prediction  problem.  Justification 
for  the  unitary  weighting  derivation  of 
the  multiple -score  distributions  ana- 
lyzed in  this  study  (Figs.  3A,  3B,  4A, 
4B,  5A  and  B5)  comes  from  several 
studies  in  the  literature  showing  that 
combining  tests  with  a variety  of 
weights,  such  as  Beta  weights,  1/ 
standard  deviation,  summed  standard 
scores,  or  even  simple  addition  of  raw 
scores  results  in  almost  identical  pre- 
dictive relationships  with  most  cri- 
teria/^’^’^ 

With  these  limitations  in  mind,  the 
task  of  integrating  the  data  from  the 
numerous  graphs  and  tables  presented 
in  the  body  of  this  paper  was  under- 
taken first  by  compiling  a summary 
table  (Table  VI)  including  comparable 
items  of  data  from  each  display. 

It  should  be  noted  at  the  outset  that 
the  major  assumptions  basic  for  the 
decisions  as  to  which  data  points  to  in- 
clude in  Table  VI  was  that  the  most 
meaningful  selection  system  would  in- 
volve an  ECOP  at  which  the  margin  of 
correct  selection  decisions  (CD's)  over 
incorrect  decisions  (ICD's),  defined  by 
the  COPV  percentage,  was  maximum. 
The  remaining  indices  were  either  cal- 
culated from  the  tables  or  taken  direct- 
ly from  the  graphs  indicated  in  the  far 
right  column  in  Table  VI.  The  follow- 
ing statements  are  indicated: 

1.  First  of  all,  for  the  EA^12  group 
(58.2%)  all  of  the  tests  singly  (except 
SP)  or  in  combinations,  provide  2-5 
times  the  selection  efficiency  as  they 


do  for  the  lesser  educated  segment  of 
the  sample  of  955  non-rated  candidates 
for  the  submarine  service . Thus  ac- 
cepting all  non-rated  volunteers  with  at 
least  a high  school  education,  irrespec- 
tive of  any  test  scores  would  result  in  a 
success  ratio  (SUR)  of  about  79%  (4th 
from  bottom  row  in  Table  VI) . This 
SUR  could  be  raised  to  85%  by  accepting 
high  school  graduates  with  GCT+MECH 
- 90  (100-FN  in  row  4 in  Table  VI) . 
Further,  it  is  seen  in  the  second  row 
from  the  bottom  of  Table  VI  that  an 
ECOP  EA<12,  Age<L8  would  yield  an 
81%  SUR  and  an  RL=  33%,  that  is  66%  of 
those  who  would  drop  would  be  rejected 
at  a "cost"  of  losing  33%  of  the  SubSchl 
graduates  (RL=  66-33). 

However,  in  an  era  when  presumably 
as  many  as  possible  of  the  available 
men  of  acceptable  quality  are  needed  for 
submarine  service  at  this  time  (i.e.,  a 
high  SER) , the  question  as  to  whether 
FP's  in  the  50%  range  can  be  tolerated 
need  to  be  weighed  prior  to  the  imposi- 
tion of  any  ECOP's  for  this  higher  EA 
group. 

2.  With  an  overall  AR  of  about  49%, 
identification  of  a useful  ECOP  for  the 
42%  who  are  in  the  EAC12  group  is  dif- 
ficult. Considering  all  the  indices,  an 
ECOP  GCT+MECH/100  may  be  helpful. 
At  least  FP  could  be  reduced  to  42% 
with  an  attrition  rate  reduced  from  49% 
to  33%  (row  3 table  VI). 


3.  Assuming  the  present  require- 
ment of  a SER  approaching  90-100%, 
the  most  effective  ECOP  resulting  in  a 
minimal  FP  (25-35%)  would  be  an  ECOP 
ARI+MECH/90-95  for  EA  group <12 
years,  (Fig.  3A). 
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Table  VI.  Comparative  Selection  Efficiency  Indices  for  Five 
Test  Scores  and  Four  Moderator  Factors 


Variables 

E.A. 

Maximum 

COPV 

% 

CD 

ECOP 
at  that 
point  % 

RL 

FN/ARa 

% 

FP 

% 

From  Fig. 
or  Table 

Tests 

GCT 

< 12 

19 

59 

50 

18 

40 

40 

2A 

£ 12 

58 

79 

40 

01 

21 

50 

2B 

GCT+MECH 

< 12 

25 

62 

100 

25 

33 

42 

3A 

:>  12 

67 

83 

90 

15 

15 

50 

3B 

GCT+MECH+SP 

< 12 

18 

59 

150 

17 

37 

45 

4A 

£ 12 

63 

81 

130 

11 

16 

36 

4B 

ARI+MECH+SP 

< 12 

13 

57 

130 

5 

42 

37 

5A 

^ 12 

62 

81 

135 

20 

16 

47 

5B 

SP 

b 

40C 

73 

35 

6 

24 

58 

6 

Moderator 

Factors 

E A 



33 

67 

12/12 

31 

21 

51 

T-I 

Age 

- 

34 

67 

17/>18 

13 

28 

51 

T-H 

EA/Age 

34 

67 

EAC12 

Age<18 

33 

19 

51 

T-IH 

Paygr  ade/E  A<  12 
E-l 

— 

33 

67 

E-l/ 

EA<12 

24 

25 

51 

T-IV 

SL 

FN=AR;  see  Appendix  A for  definition  of  terms. 

^EA,  Educational  Achievement  not  controlled  in  this  analysis. 
CMaximum  COPY  was  at  ECOP  SP/35  but  only  based  upon  N=  4. 
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4 . At  least  a partial  answer  to  the 
question  "Should  a floor  for  combined 
scores  or  for  one  or  more  individual 
scores  (especially  GCT)  be  established" 
emerged  from  the  results  of  this  study. 
For  high-school  graduates,  there  does 
not  appear  to  be  any  single  or  multiple 
score  points  that  would  significantly 
improve  the  selection  efficiency  for  this 
E A group.  For  the  high-school  drop 
group  on  the  other  hand,  an  ECOP  at 
GCT+MECH=  90  may  be  useful  (Fig. 

3A) . An  approximately  equivalent 
ECOP  for  this  EA  group  would  be  ECOP 
GCT=  50  (Fig.  2A).  Adding  SP  either 
to  GCT+MECH  (Figs.  4A,  4B)  or  to 
ARI+MECH  (Figs.  5A,  5B)  doesn't  ap- 
pear to  significantly  improve  the  situ- 
ation. 

5.  Should  SP  and/or  ARI+MECH+SP 
scores  be  deleted  as  one  of  the  eligibil- 
ity criteria  options  Figure  6 presents 

; the  graphs  bearing  on  the  first  part  of 
this  question,  namely,  the  use  of  SP 
scores  as  criteria.  As  seen  in  Table 
VI,  5th  row  from  the  bottom,  SP  is 
most  efficient  in  the  40-45  score  range 
(COPV  46-37%),  but  FP  is  about  60%  in 
this  range  signifying  that  60%  of  those 
rejected  by  an  ECOP  in  that  range 
would  have  graduated  from  SubSchl . 

Too,  taken  together  with  the  fact  that 
RL=  6%  in  that  range  (i.e. , only  6% 
more  of  potential  failures  than  potential 
graduates  would  be  eliminated  by  an 
ECOP  in  this  score  range)  it  appears 
that  the  SP  score  alone  has  little  po- 
tentiality with  the  non-rated  segment  of 
the  SubSchl  input. 

6.  As  for  the  summed  score,  ARI+ 
MECH+SP  within  the  EA=12  segment,  as 
with  other  test  scores  and  test  score 
combinations  for  this  EA  group,  does 


not  demonstrate  any  fruitful  ECOP  pos- 
sibilities since  SUR  is  over  80%  re- 
gardless of  these  combined  scores. 

For  the  EA<12  group  however,  looking 
at  Fig.  5A  within  the  score  range  145- 
165,  AR  drops  from  43  to  24%  while 
FP  holds  at  about  50%  in  this  range . 

This  means,  for  example,  that  with  an 
ECOP  at  ARI/MECH/SP/equals  155,  and 
assuming  that  the  personnel  situation 
allows  for  acceptance  of  only  the  1/3 
above  ECOP/155,  the  attrition  rate 
would  be  only  35%  which  is  14  percent- 
age points  below  the  existing  AR  of  49% 
across  the  total  group  of  high  school 
drops  (Table  I) . 

7 .  Another  question  pertains  to  the 
usefulness  of  the  GCT+MECH+SP  within 
the  hypothetical  ECOP  141-150  range 
(combined  score  = 150  with  9 point 
waiver  allowed) . Looking  first  at  Table 
VI  row  6 and  Fig.  4B,  any  ECOP's  are 
impractical  in  a high  SER  situation, 
since  for  all  points  above  ECOP/135, 

FP  is  greater  than  50%  meaning  of 
course  that  the  majority  of  those  re- 
jected would  have  graduated.  However, 
turning  to  Fig.  4A  and  row  5 in  Table 
VI,  the  selection  efficiency  peaks  at 
ECOP/150  and  declines  within  the  9- 
point  waiverable  range.  Translated  to 
outcomes,  if  a rigid  ECOP  at  GCT+ 
MECH+SP=  150  were  to  be  adopted  for 
the  EA<12  segment  of  the  non-rated 
input,  the  following  approximate  conse- 
quences would  be  expected:  (1)  Forty- 
five  percent  would  be  rejected  (A+C  in 
Fig.  1);  (2)  of  these,  45%  (FP)  or  20% 
of  total  input  would  have  graduated ; 

(3)  of  the  55%  accepted  37%  would  drop 
in  SubSchl  (about  20%  of  total  input); 
and,  finally  (4)  this  results  in  a success 
ratio  (SUR)  of  about  35%  as  compared 
to  51%  (Table  I)  for  the  high  school  drop 
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group  as  a whole  irrespective  of  test 
scores. 

8.  Overall,  the  most  useful  ECOP 
for  the  EA<  1 2 group  appears  to  be  E CO  P 
GCT+MECH/95  (See  Fig.  3A).  Thus, 
imposing  this  ECOP  would  reject  about 
21%,  of  which  34%  (FP=34%)  (7%  of 
total  input)  would  have  graduated. 
Further,  of  the  79%  accepted,  61% 
should  graduate.  In  sum,  a SUR  of  48% 
would  be  realized.  While  this  is  only 
3%  less  than  the  present  overall  SUR  of 
this  EA  group  (51%)  the  resource  cost 
of  processing  the  21%  rejected  by  this 
ECOP  would  be  avoided. 

9.  Finally,  the  question  of  a possi- 
ble "bottom-out"  GCT  score  presum- 
ably indicative  of  limited  literacy  on  the 
part  of  the  candidate  has  been  men- 
tioned in  the  context  of  the  discussion  of 
Fig.  2A.  Apparently,  a useful  ECOP 
for  the  GCT  distribution  obtained  from 
the  segment  of  the  non-rated  input 
whose  EA  is  512  years  cannot  be  mean- 
ingfully identified  since  the  attrition 
rate  of  this  group  is  about  21%.  How- 
ever, by  means  of  a backward  extrapo- 
lation of  the  FP  curve  in  Fig.  2A, 
something  of  the  order  of  GCT  = 33  ap- 
pears to  approximate  that  "bottom" 
point  for  non-rated  men  without  a high 
school  diploma.  However,  with  ap- 
proximately 2.5%  of  the  EAC12  input 
segment  having  GCT  scores  .5.40  and  an 
expected  failure  rate  of  70-80%  (i.e. , 

FP  = 20-30%),  a more  realistic  cut-off 
point  might  be  GCT=  40. 
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APPENDIX  A 


AR  - Attrition  Rate  or  failure  rate  in  a 
given  personnel  endeavor,  e.g. , 
Submarine  School  drop  rate  for 
those  accepted  by  a given  ECOP 
(see  below).  [AR=  D/B+Dj-100*. 
AR=FN. 

CD  - Percent  of  Correct  Decisions, 
calculated  as  follows:  CD=  [B+ 
C/N]  -100. 

COPY  - Coefficient  of  Practical  Valid- 
ity is  the  difference  between 
CD's  and  ICD's  changed  to 
percent  of  the  total  selection 
decisions  (N).  Thus  (Fig.  1) 
COPV=[B+C]  - [A+Dl/N‘100. 
This  index  is  essentially 
equivalent  to  the  index  of  Se- 
lection Efficiency  (S)  proposed 
by  Jenkins  (Ref.  No.  20).  In 
a selection  system  with  SER 
30-70%,  COPV  apparently 
approximates  numerically  the 
coefficient  of  correlation  (op. 
cit.). 

EA  - Educational  Achievement,  the 

number  of  years  of  formal  educa- 
tion (not  including  USN  Technical 
Schools)  completed.  Ordinarily 
completion  of  high  school  results 
in  EA=12  and  receipt  of  a bach- 
elors degree,  EA  = 16,  etc. 

ECOP  - Experimental  Cut-off  Point,  a 
hypothetical  point  on  a selec- 
tion test  score  distribution, 
which  is  used  as  a basis  for 
the  acceptance/rejection  de- 

* Most  of  these  terms  are  operationally  defined  with 

reference  to  tke  schematic  in  Fig.  1,  p.  3a. 


cisions  required  by  the  personnel 
situation. 

FN  - False  Negatives , those  candidates 
accepted  by  a given  ECOP  who 
subsequently  fail  e.g.  drop  in 
SubSehl,  quadrant  D in  Fig.  1. 
FN=[D/B+D]  *100.  FN  is  equiva- 
lent to  AR . 

FP  - False  Positives,  those  candidates 
rejected  by  a given  ECOP  who 
would  have  succeeded  e.g.  grad- 
uated from  SubSehl  (Quadrant  A) . 
FP=[A/A+C)  -100. 

ICD  - Incorrect  Decisions  resulting 
from  the  application  of  a given 
ECOP.  ICD=FP+FN=A+D. 

Moderator  Variable  - A psychometric 
term  defined  as  a variable  used 
as  a basis  for  identifying  sub- 
groups within  a given  population 
sample,  the  assumption  being 
that  different  predictive  relation- 
ships exist  within  the  subgroups 
so  delineated  } 4 

RL  - Relative  Losses,  that  is  the  dif- 
ference between  the  percent  of 
those  who  would  fail  that  would 
be  rejected  by  an  ECOP  and  the 
percent  of  those  who  would  suc- 
ceed that  would  also  be  rejected 
by  the  same  cut-off.  RL=  [(C/ 
C+D)  - (A/A+B)]  *100. 

SER  - Selection  Ratio  is  the  ratio  of 
number  of  selectees  required 
to  the  number  available  (usu- 
ally presented  in  percentage 
terms).  In  a sense,  this  index 


A-l 


is  analogous  to  the  econometric 
demand/supply  ratio  SER= 
(B/N).100. 

Skewness  - Degree  of  departure  from 
symmetry  of  a frequency 
distribution.  Positive 
skewness  indicates  reduced 
frequency  density  at  the 
high  end  of  the  scale;  nega- 
tive skewness  means  low 
density  on  the  opposite  end. 


SUP.  - Success  Ratio,  that  is  the  per- 
centage of  those  accepted  (B+D) 
who  succeed.  SUR=[B/B+D]  * 
100.  Also  SUR  is  the  comple- 
ment of  FN.  SUR=  100-FN. 
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